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ABSTRACT 



An overhead scanning system records pages from bound 
documents in an upright and open condition. A light stripe 
is projected onto each page of the bound document to 
measure the non-planar deformations of each page. The 
scanning system records a first image with the light stripe 
and a second image without the light stripe. After identifying 
a set of pixels representing the light stripe in the first image, 
the set of pixels is smoothed to minimize large transitions 
between scan lines. Subsequently, a crease is identified in the 
set of pixels that represents the spine of the bound document. 
The pixels forming the crease are replaced with pixels that 
are defined using a parametric model that approximates the 
spine region of a bound document. This substitution insures 
that the spine region of a bound document is accurately 
depicted before determining a page shape transform with the 
identified set of pixels representing defining the projected 
light stripe. An image correction system de-warps the sec- 
ond image using the page shape transform and a perspective 
transform to correct distortion due to the pages of the bound 
document being curved relative to the support plane of the 
system. The de-warped image is determined by "polling" 
location in the second image for output values. 

28 Claims, 13 Drawing Sheets 
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METHOD FOR DETERMINING THE 
PROFILE OF A BOUND DOCUMENT WITH 
STRUCTURED LIGHT 

The present invention relates generally to a system for 
scanning bound documents, and more particularly to a 
method for determining the height profile of a bound docu- 
ment positioned in an open and face-up condition. 

BACKGROUND OF THE INVENTION 

Book scanners are becoming more prevalent given the 
advent of inexpensive digital storage and the ease with 
which digital information can be disseminated and repro- 
duced. Recording bound documents in a digital formal has 
become an important manner in which to preserve informa- 
tion in ancient books, periodicals, manuscripts, or any other 
document having a bound edge with a hard or soft cover. 
Institutions such as the Library of Congress, the Vatican 
library, and university libraries seek to scan large numbers of 
volumes of books, both to create digital libraries capable of 
feeding the information superhighway and to halt the rapid 
decay of information stored in older printed works. Once the 
content of a page of a bound document is scanned and 
electronically recorded, the recorded digital image can then 
be manipulated or processed to enhanced forms not origi- 
nally available in its bound format. For example, digital 
images recorded on a storage medium can be reproduced 
audibly using applications that generate speech from ASCII 
text. Alternatively, bound documents digitally recorded can 
be reformatted and supplemented with additional informa- 
tion and reproduced on demand as hardcopy using an 
electronic printing machine or the like. 

Different arrangements exist for scanning bound docu- 
ments. Most arrangements are traditional flat-bed platen 
scanners that scan bound documents in a face-down posi- 
tion. A disadvantage of flat-bed scanning arrangements is 
that they require the application of a force to the spine region 
of a bound document to insure that the bound document 
comes within a scanner's depth of focus. Besides damaging 
the spine region of a bound document, the scanning process 
is tedious and time-consuming because the bound document 
must be repositioned after scanning each page. In addition, 
image quality is often poor due to loss of focus, uneven 
illumination, and distortion caused by curvature of a page in 
the vicinity of the binding. U.S. Pat. Nos. 4,980,720 and 
5,276,530 to Siegel disclose an improved arrangement that 
minimizes the application of force to the spine region of a 
bound document and improves the image quality of scanned 
images. Specifically, Siegel discloses a flat-bed scanner with 
a book mode that minimizes the force typically applied to 
the spine region of a bound document. In the book scan 
mode, a page height detector mounted on a scan carriage 
accounts for the deviation of a book page from the horizontal 
platen plane in the vicinity of a book's binding. In addition, 
electronic correction of the captured image is used to 
compensate for loss of focus, image compression, and 
uneven illumination. Besides electronic correction, the qual- 
ity of captured images is maximized by modifying mechani- 
cal parameters such as scan rate, and local light source 
intensity. 

Other face-down scanning arrangements minimize the 
stress on the spine region of bound documents by forming a 
wedge or an angled platen. Examples of wedge platens for 
scanning bound documents include, U.S. Pat. No. 5,359,207 
to Turner and German Patent DE-A1 3546404. Turner dis- 
closes a book scanner with contact imaging that employs 
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two-dimensional sensor arrays that are attached at a com- 
mon edge and disposed in a wedge shaped manner. German 
Patent DE-A1 3546404 discloses a roof-shaped book support 
apparatus for use in a copier application. The optics of the 
s roof-shaped copier arrangement provide that opposing pages 
of a bound book are simultaneously imaged in the same 
image plane. 

In alternate arrangements, platenless scanners capture 
image data with a document in a face -up position. Such 

10 arrangements do not require additional stress to be applied 
the binding region of a document when scanned since the 
document is scanned in its natural open condition. For 
example, platenless scanners that record a bound document 
opened face-up from above are disclosed in U.S. Pat. Nos.: 

]5 5,084,611; 5,377,019; and 5,416,609. Specifically, U.S. Pat. 
No. 5,084,611 discloses a document reading apparatus 
which is disposed above a document table. Curves of the 
bound document are detected using a curvature detector 
which includes a linear light source. In addition, the linear 

20 light source defines a brightness distribution which is used 
to rectify image signals received from the document reading 
means. U.S. Pat. No. 5,377,019 discloses a reading appara- 
tus which can determine an effective image pickup area 
containing objects such as an operator's hand. U.S. Pat. No. 

25 5,416,609 discloses an image pickup apparatus with a focus 
adjusting means for controlling an imaging optical system so 
that an image is in focus on a line sensor during scanning. 

In either face-down or face-up scanning arrangements, 
image profile measurements must be precise so that image 

30 distortion caused by recording non-planar original images 
can be accurately corrected. Linear light stripping is one 
method for measuring the profile of a non-planar original 
image, t he basic principles of which are described by Ballard 
et al. in "Computer Vision", Prentice-Hall, 1982 pp. 52-54. 

35 Ballard et al. also discloses other methods for measuring the 
profile of a non-planar original image such as laser based 
spot range finders and ultrasonic range finding. With each of 
these range finding techniques geometrical information can 
be used develop a profile of a non-planar document. Once a 

40 profile of a non-planar original image is developed, original 
image data is de-warped so that it appears planar. Some 
de-warping techniques transform recorded image data using 
image processing routines that correct skew, and compres- 
sion or elongation caused by oblique viewing angles. Other 

45 de-warping techniques can be performed by adjusting dif- 
ferent components of a scanning assembly. In the case of 
linear light stripping, the profile, which is measured as a 
grayscale image of a light stripe projected across a bound 
document, can be used to define a transformation to de-warp 

50 recorded image data. 

A scanning system that determines the geometrical shape 
of a non-planar surface of a bound document from an image 
with a light stripe projected thereacross may not always 
develop an accurate profile of the non-planar surface. One 

55 source of inaccuracy is self-illumination which results when 
light from the light stripe is reflected onto other parts of the 
bound document. The reflections cause noise in the recorded 
image that must be distinguished from the light stripe 
projection. Another source of inaccuracy is the aspect ratio 

60 of a light stripe projector. The higher the aspect ratio the 
more area the light stripe illuminates a page. This may 
causes inaccuracies when abrupt transitions along the light 
stripe blur together. Yet another source of inaccuracy is 
noise. A further source of inaccuracy results when printed 

65 marks on a page of a bound document do on reflect light 
from the light stripe projector and cause dropouts along a 
recorded light stripe. Dropouts may become a significant 
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source of error when they fragment the recorded light stripe FIG. 1 is an illustration of a system for scanning bound 

into segments. Even though thresholding may eliminate documents incorporating the present invention; 

many sources of noise, the geometrical shape of a bound FIG. 2 is a block diagram of the system architecture of the 

document may not eliminate noise resulting from a very scanning system shown in FIG. 1; 

sharp spine region. Given these sources of inaccuracy, it S FIG 3 ^ a flow diagram showing the general steps for 
would be desirable to provide a scanning system that can acquiring image data of a bound document using the scan- 
accurately detect a light stripe in an image recording a light ning system incorporating the present invention; 
stripe projected across a non-planar bound document. Once FIG. 4 is an illustration of the calibration system of the 
the geometrical shape of the non-planar surfaced of a bound scanning system* 

document is determined, the scanning system can transform 10 ' RG 5 fe a ^ di showj ^ { fof 

non-planar image data to appear planar. calibrating the scanning system shown in FIG. 1; 

SUMMARY OF THE INVENTION PIG - 6 shows a light stripe projected across a bound 

document in accordance with the present invention; 

In accordance with one aspect of the invention, there is ^ FIG. 7 is a flow chart of the steps for determining a page 

provided a method for determining a shape of the bound shape transform (T 2 ) with a first image (I J which is a 

document positioned in an imaging area of a scanning recording of a bound document with a light stripe projected 

apparatus having an image acquisition system for recording across it' 

images and a light stripe projector for projecting a light FIG ^ [ s a graph of the trace of a light stripe; 

stripe. The method includes the steps of: recording, with the FIGS % ^ and u illustrale lookup tables thal define a 

image acquisition system, an image of the bound document sh transform (y ) thal corre sponds to the trace of a 

having a light stripe projected thereacross by the light stripe ^ s(rf shown in FIG 8; 

projector, the image being defined by an array of scan lines, .„ . . . , . , 

v . , / 1. . • 1 ■ . . . • 1 FIG. 12 illustrates a geometry that can be used to track a 

with each scan line having pixels with an intensity value; . °.. . J t ... , t 

. . . , ■ , <- • ,i ■ / location in a world coordinate system while determining a 

adaptively thresholding the pixels forming the image lo . , \ e c ,. Ut 

. , . 1 j * • jf * i_ 1 ■ u < • »u 1 , * 25 location in a page coordinate system tor a trace or a light 

identity pixels denning part of the light stripe, the adaptive . ^ r to ' to 

thresholding step compensating for non-uniform illumina- " J? w ' . , , r 

tion across the bound document recorded by the image FIG. 13 is a flow chart detailing the steps for de-warping 

acquisition system; identifying a pixel starting location that ^corded image using a perspective transform and a page 

forms part of the light stripe on a first scan line near a sha P e transtorm - 

midpoint of the image; tracking, from the pixel starting 30 DETAILED DESCRIPTION 

location, a first set of pixels that further define the light stripe Referring now to the drawings where the showings are for 

in the image, the set of pixels representing a trace of the light lhe purpQSe of describing tne pre ferred embodiment of the 

stripe projected across the bound document; smoothing the mven tion and not for limiting same, FIG. 1 shows a bound 

first set of pixels to minimize large transitions between scan 35 document scarm ing system 4 incorporating the present 

lines; detecting a crease location along the first set of pixels invention . Generally; the bound document scanning system 

that identifies a spine region of the bound document; and 4 includes a p i at form 8 for supporting a bound document 10 

responsive to the detecting step, modifying the first set of in an open conditiori( The p i atform g 1S supported by legs 12. 

pixels representing the trace, the modifying step replacing By providmg removable or foldable legs 12, the platform 8 

pixels near the crease location with a second set of pixels ^ ifi portab]e and c£m be readily transported frora one location 

that are determined using a parametric model that approxi- , Q anotner BuiU into platform 8 is a calibration system 

mates the spine region of the bound document. which includes marks 15> the calibration system being 

In accordance with another aspect of the invention, there generally designated by reference numeral 14. In addition to 

is provided a method for scanning a sequence of non-planar the platform 8 and the calibration system 14, bound docu- 

original images of a bound document. The method includes 45 me nt scanning system 4 includes an image acquisition 

the steps of: recording geometry data and raw image data for system 20, a light stripe projection system 22, an electronic 

each image in the sequence of non-planar original images, subsystem system 24 (ESS). Each of the elements of the 

the recording step including the steps of: projecting a light scanning system 4 are not positioned in an assigned location 

stripe on a non-planar original image, acquiring geometry relative to each other. When the scanning system 4 is 

data with an image acquisition system, extinguishing the 5Q positioned in a new location, the calibration system 14 is 

light stripe, and acquiring raw image daUi with the image used to orient the image acquisition system 20 relative to the 

acquisition system; correcting the recorded raw image data platform 8. 

for each image in the sequence of non-planar original Image acquisition system 20 in a preferred embodiment is 

images with the recorded geometry data, the correcting step a high resolution digital color camera having internal optics, 

performing, for each image in the sequence of non-planar 5S shown generally by reference numeral 21, that focus an 

original images, the steps of: determining a page shape image on internal focal plane 23. The high resolution camera 

transform with the geometry data, and de-warping the contains an array of active photosensors that convert optical 

recorded image data with the page shape transform to image data int0 electrical signals. Photosensor arrays are 

provide corrected image data. weU known in the arl and include two-dimensional fiat panel 

60 detectors and one-dimensional and two-dimensional charge 
coupled devices (CCDs). An example of a flat panel detector 

These and other aspects of the invention will become is disclosed in U.S. Pat. No. 5,017,989, and an example of 

apparent frora the following description which illustrates a scanning carriage system using a one-dimensional CCD 

preferred and alternative embodiments of the invention read array is disclosed in U.S. Pat. No, 5,276,530, the disclosures 

in conjunction with the accompanying drawings wherein the 65 of which are incorporated herein by reference, 

same reference numerals have been applied to like parts and Light stripe projection system 22 is used by the scanning 

in which: system 4 to measure the surface shape of bound document 



BRIEF DESCRIPTION OF THE DRAWINGS 
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10 by projecting across the bound document 10 on platform sions. Page shape transform generator 40 uses stored page 

8 one or more stripes of light that are identified by numerals shape data 34 to provide a page shape transform (T 2 ). The 

5 and 6 in FIG. 1. The basic principles of light striping are page shape transform (T2) defines an interrelationship 

described by Ballard et al. in "Computer Vision", Prentice- between the world coordinate system and a page coordinate 

Hall, 1982 pp. 52-54. Light striping is a type of structured s system which defines an output space in which a page of 

light in which the illumination of an object in a scene is used bound document 10 is represented in two dimensions. The 

to extract geometrical information about the object itself. image correction system 42 generates corrected image data 

Here, light striping is used to identify the shape of bound 44 from image data 32 using the perspective transform (Tj) 

document 10 on platform 8. More specifically, light striping and the page shape transform (T2). The corrected image data 

is performed in order to detect the three-dimensional shape 10 44 is either stored on storage device 28 or output directly to 

of a page and thereby correct distortions cause by image a peripheral device through application driver 46. Applica- 

perspective, skew, and compression or elongation intro- lion driver 46 transmits corrected image data 44 to any 

duced by oblique viewing angles of image acquisition peripheral device adapted for displaying, storing, or repro- 

system 20. The light-stripe source 22 can contain a laser or ducing as hardcopy corrected image data 44. 

an ordinary light bulb that projects an image of a slit. It ]5 FIG. 3 is a flow diagram showing the general steps for 

should be understood, however, by those skilled in the art acquiring image data with the bound document scanning 

that there exists a number of different methods which can be system 4. At step 102, the scanning system 4 is set up as 

used to measure the surface shape of a bound document. In shown in FIG. 1. At step 104, a system operator positions the 

alternate embodiments, the three-dimensional surface of a image acquisition system 20 to point al an imaging area, 

bound document can be measured using stereo vision, laser 20 which is defined herein as an area that is recorded on internal 

range-finding, sonar rangefinding, projection of a grid, or an focal plane 23 of image acquisition system 20. 

alternate mechanical means. Subsequently, if necessary the system operator at step 106 

After recording a digital representation of light stripes adjusts the direction and zoom of image acquisition system 

formed across the bound document 10 using image acqui- 20. At step 108, ESS 24 calibrates the image acquisition 

sition system 20, ESS 24 analyzes the digitally recorded 2 s system 20 to the imaging area defined at steps 104 and 106 

image daia to identify the shape of the open pages of bound with a perspective transform (TJ. After system calibration 

document 10. A microprocessor and control software, which at step 108, the system operator places bound document 10 

forms ESS 24, coordinate the general operational units of on platform 8 in the imaging area at step 110. At step 112, 

document scanning system 4. ESS 24 which communicates light stripe projection system 22 is turned on. At step 114, an 

with calibration system 14, image acquisition system 20, and 30 image (Ij) of a light stripe projected across bound document 

light stripe projection system 22, can either be integral with 10 is acquired by image acquisition system 20. At step 116, 

image acquisition system 20, as shown, or stand alone as the light stripe projection system 22 is turned off. At step 

part of a computer workstation or the like. 118, an image (I 2 ) of the bound document 10 is acquired 

FIG. 2 shows a general block diagram of the system without the projection of a light stripe across it. At step 120, 

architecture of scanning system 4. When operating, scanning 35 if there is a new page to be scanned, steps 112, 114, 116, and 

system 4 is in one of four states: calibrating, determining 118 are repeated. 

page shape information, scanning image data, or de-warping With continued reference to FIG. 3, at step 122 the image 

image data. The system controller 26, which forms part of processing system 36 determines a page shape transform 

ESS 24, coordinates operations of calibration system 14, (T 2 ) of the current page shape using the image (I a ) acquired 

image acquisition system 20, and light stripe projection 40 at step 114. At step 124, the image (12) is de -warped to form 

system 22. In operation, depending on the current state of the a corrected image (I 3 ) using the perspective transform (T,) 

scanning system 4, system controller 26 selectively invokes determined at step 108, and the page shape transform (T 2 ) 

calibration system 14 or light stripe projection system 22. determined at step 122. Steps 122 and 124 are repeated at 

When invoking the calibration system 14, either calibration step 126 if a subsequent page is acquired at steps 112, 114, 

object 18 or movable tabs 16 are put in place (as shown in 45 116, and 118. It will be understood by those skilled in the art 

FIG. 4), and when invoking the light stripe projection that the steps 112, 114, 116, and 118 can operate in sequence 

system 22, one or more light stripes 5 and 6 are obliquely or in parallel with steps 122 and 124 depending on the 

projected across platform 8. Digital image data received by performance requirements or the hardware constraints of 

system controller 26 from image acquisition system 20 is scanning system 4. At step 128, if the system operator 

stored on a storage device shown generally by reference 50 changes the direction or zoom, step 106 is repeated instead 

numeral 28. The storage device 28 may include RAM, flash of step 112. Alternatively, if a new imaging area is selected 

memory, floppy disk, or another form of optical or magnetic by the system operator at step 130, step 104 is repeated. At 

storage. Depending on whether calibration system 14 and step 132, if no more pages are to be scanned, system 4 

light projection system 22 are invoked, digital image data remains in a ready state for scanning subsequent bound 

received by system controller 26 from image acquisition 55 documents. 

system 20 is stored on storage device 28 either as calibration In accordance with the present invention, high-speed 

data 30, image data 32, or page shape data 34. scanning of bound documents is enhanced by delaying 

Once stored on storage device 28, image data is manipu- ' electronic perspective correction of recorded images. Since 

lated by image processing system 36 which includes a a geometrical profile map (i.e. page shape data) and raw 

perspective transform generator 38, a page shape transform 60 image data of an image is recorded at steps 114 and 116 

generator 40, and an image correction system 42. Perspec- independent from steps 122 and 124, image correction can 

tive transform generator 38 uses image calibration data 30 to be delayed or computed in parallel with image acquisition 

provide a perspective transform (T a ). The perspective trans- operations. For example, image correction at steps 122 and 

form (T a ) defines an interrelationship between a two- 124 can be delayed until such lime as the bound document 

dimensional image coordinate system of the image acquisi- 65 is to be viewed or rendered. In addition, when computational 

lion system 20 and a world coordinate system in which a correction of raw image data is delayed until the raw image 

page of a bound document is represented in three dimen- data is read (e.g. displayed on a terminal or reproduced on 
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hardcopy), the overhead of image correction may be hidden At step 144, light stripe 50 is projected at an oblique angle 

from the reader by preparing additional pages of raw image by light stripe projection system 22 across calibration block 

data while a current page is read. 18 which is of a known height. During step 144, system 

FIG. 4 shows calibration system 14 of the bound docu- controller 26 (shown in FIG. 2) directs the operation of 

ment scanning system 4 in detail. In general, two different s calibration system 14 and light stripe projection system 22. 

elements of scanning system 4 require calibration. First, the Specifically -at step 144, system controller 26 records a 

image acquisition system 20 must be calibrated relative to second calibration image (I c2 ) with image acquisition sys- 

the platform 8 in order that locations in the three dimen- tern 20 and stores it as calibration data 30. For the purpose 

sional space of the platform 8 can be identified in an image of describing the invention, the scanning system 4 is set up 

recorded along the focal plane 23 of image acquisition ](J such that the projection of the light stripe 50 on the platform 

system 20. Second, the image acquisition system 20 and the 8 falls along a single horizontal line of pixels (u) defining the 

platform 8 must be calibrated relative to the light stripe internal focal plane 23 (u,v) of image acquisition system 20. 

projector 22 in order that the height of an object on the It will, however, be understood by those skilled in the art that 

platform 8 can be determined based on the position of a light m js assumption can be relaxed so that a bound document 

stripe in an image recorded by the image acquisition system may t, e positioned so that the projection of light stripe 50 

20. To perform the calibration of two different elements of faUs a]ong severa] horizontal lines of pixels defining the 

scanning system 4, the calibration system 14 includes call- internal focal p]anc 23. ]n the case of the light stripe falling 

bration marks 15 i on the surface of platform 8, and a a i ong a single horizontal line of pixels, the vertical position 

calibration object 18. 1 he calibration marks 15 are spaced at (y) f ^ ^ horizontal line of ixels (al u} in lhe 

a predetermined interval from each other on platrorm 0 to > ; , r t 1 n u a a a u • ■ 

define a support plane (X,Y). ft will be understood, however, 20 inte ™ focal pi™ 23 can be defined as v, where v is in 

by those skilled in the art that the calibration marks 15 can un >* of « P ixels ' ln * ls exam P K le ' ^ light stripe 50 is 

be positioned so as to define a support surface having a calibrated by placing calibration object 18, which is of a 

non-planar shape such as a wedge. At least two of the known hei S ht h ~> in lhe P ath ot ,he h £ ht stn P e 50 ™ e 

calibration marks 15 are positioned outside the X,Y support ima g e acquisition system 20 then records the position ol the 

plane of platform 8 to make calibration calculations more 2 5 P ro J eclion of the li Z hl slri P e 50 on the t0 P ot calibration 

numerically stable. The two calibration marks 15 which are ob J ect 18 - If the hei S ht of the calibration object 18 is defined 

outside the support plane of platform 8 appear on movable in the internal focal P lane 23 as V ^ then the hei S hl h /> al a 

tabs 16 that rotate out along side platform 8 when not in use, P oint on an arbitrary object, such as a bound document, at 

as indicated generally by arrows 17. The calibration object which the U g hl stri P e 50 projected Ihereon is viewed at an 

18 when not in use is either removed from the surface of 30 ima S e scan line v *> 15 S^en by: 
platform 8, or alternatively retracted into its surface. Once 

calibration is complete, a perspective transform (TJ is /jj> = (v* - v^) — 

provided by perspective transform generator 38 (shown in Vuh ~ Vls0 

FIG. 2). The perspective transform (T A ) defines a transform It will be understood by those skilled in the art that the light 
between a world coordinate system (X,Y,Z), indicated gen- 35 stripe calibration can be accomplished in many different 
erally by the reference numeral 53, which represents a three ways. The purpose of light stripe calibration is to infer the 
dimensional space of bound document 10 on platform 8, and height of each point in a bound document 10 above the 
an image coordinate system (u,v), indicated generally by surface of platform 8 when it is recorded on the internal 
reference numeral 55, which represents image pixels (or focal plane 23 of the image acquisition system 20. 
image data) in internal focal plane 23 in two dimensions. 40 FIG. 6 shows a light stripe 50 projected across bound 
FIG. 5 is a flow diagram that details the steps for document 10. When recording pages 11 and 13 of bound 
performing calibration step 108 shown in FIG. 4. Generally document 10 two images are recorded by image acquisition 
at step 108, a perspective transform (TJ is determined system 20. As noted in FIG. 3 at steps 114 and 118, a first 
which calibrates the current imaging area defined at steps image (I.,) records bound document 10 with the light stripe 
104 and 106 in the image coordinate frame (u,v) with respect 45 projected across it and a second image (I 2 ) records bound 
to the world coordinate frame (X,Y,Z). More specifically document 10 absent the light stripe. The first image (I 3 ) is 
step 108 includes steps 140, 142, and 144. At step 140, a first used by page shape transform generator 40 to determine a . 
calibration image (I cl ) is recorded which includes calibra- page shape transform (Tj). The page shape transform (T 2 ) 
lion marks 15 shown in FIG. 4. Step 140 is performed with defines a relationship between the world coordinate system 
the calibration marks 15 visible on movable tabs 16, but 50 53 (X,Y,Z) and a page coordinate system (page x, page y), 
without positioning the calibration block 18 on platform 8, indicated generally by the reference numeral 54. The page 
During step 140, system controller 26 acquires the first coordinate system 54 (page x, page y) defines a two- 
calibration image (I cl ) from image acquisition system 20 dimensional space which is used to represeni a virtual page 
and stores it as calibration data 30 (shown in FIG. 2). At step 52 of the bound document 10 defined in the three- 
142, perspective transform generator 38 computes a per- 55 dimensional world coordinate system 53 (X,Y,Z). The vir- 
spective transform (TJ with calibration image (Icl) using tual page 52 may, for example, represent either an image 
standard calibration techniques. For example, the image displayed on a video screen or a hard copy rendering by an 
acquisition system 20 can be calibrated using the technique electronic printing system. The second image (I 2 ) is used by 
disclosed by Strat in "Recovering the Camera Parameters image correction system 42 to determine a corrected image 
from a Transformation Matrix," published in "Readings in 60 (I 3 ). 

Computer Vision: Issues, Problems, Principles, and FIG. 7 is a flow chart of the steps for determining a page 
Paradigms" Morgan Kaufmann Publishers Inc., 1987, the shape transform (T 2 ) with the first image (I : ) determined at 
pertinent portions of which are incorporated herein by step 122 shown in FIG. 3. The page shape transform (T 2 ) 
reference. Once determined, perspective transform (T^) pro- provides an estimate of the shape of bound document 10 
vides a mapping between any point in the world coordinate 65 from light stripe 50. In analyzing the light stripe in the first 
system (X,Y,Z) and any point in the image coordinate image (I 2 ) the following factors should be taken into con- 
system (u,v). sideration: the light stripe 50 is not the only visible object in 
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the image (Ij); the intensity of the light stripe 50 and the light stripe projector 22. The pre-filtering step would take 
intensity of the background of the bound document 10 vary place before the recorded image (Ij) is adaptively thresh- 
across the image ([J; the light stripe 50 is several pixels olded at step 152. Alternatively, known image processing 
wide; the edges of the light stripe 50 arc ragged and techniques that identify different color intensities can be 
sometimes broken in the image due to the light stripe 5 substituted for the pre-filtering step. This image processing 
projecting onto printed characters on pages 11 or 13 of step initially thresholds from the recorded image (1^) any 
document 10; the light stripe 50 becomes very steep in the light not having the wavelength projected by the light stripe 
spine region 9 of bound document 10; and there may exist projector 22, before the image (lj) is adaptively thresholded 
spurious bright spots in image not created by the light at step 152. Image contrast can also be increased by emitting 
stripe. 10 infra-red (1R) light with the light stripe projector 22. A 
While taking into account the above-noted factors, the projected IR light stripe can be recorded with a sensor 
following steps are performed to determine the page shape sensitive to IR light. Alternatively, a projected IR light stripe 
of the bound document 10. For simplicity, the following can be detected using a sensor sensitive to both infra-red 
description assumes that the projection of light stripe 50 on light and visible light with an IR filter mounted in front of 
platform 8 is perpendicular to the spine 9 of bound document 15 the sensor. 

10, It will be understood, however, that this method for At step 154, a starting point on the light stripe near a 

detecting a light stripe can be performed even if the path of horizontal midpoint of the bound document 10 is located, 

light stripe 50 is not perpendicular to the spine 9 of bound The horizontal location of the starting point is defined as the 

document 10. In general, image (I 2 ) represents a profile of horizontal midpoint of the image plus or minus a fixed 

the page shape of the bound document 10. More specifically, 20 horizontal distance of approximately 20 pixels. The vertical 

the profile given by the light stripe 50 is a cross-section location of the starting point, which is determined using an 

along an axis perpendicular to the spine 9 of the bound iterative process, is the vertical centroid of non-zero pixels 

document 10 . The following steps are performed to identify within a window of pixels in the image (I^. The vertical 

the light stripe 50 in image (IJ which is then used to define upper and lower dimensions of the window initially extend 

the shape of bound document 10 to correct distortion of the 25 approximately 20 pixels, toward the vertical midpoint, from 

second image (I 2 ). At step 150, an image recorded by the top and bottom of the image (1^), respectively. These 

image acquisition system 20 is located by page shape dimensions, which define a window region, are refined using 

transform generator 40 in memory 28 as stored page shape the iterative process to identify a portion of the light stripe 

data 34. In the recorded image (IJ the light stripe 50 is in the image (I J along the vertical dimension. A first step of 

positioned to be generally aligned with the horizontal scan 30 this iterative process is to compute the vertical centroid of 

lines of the image (IJ. A horizontal scan line is defined the no n- zero pixels within the window region of the image 

herein as a scan line in image (Ij) that is perpendicular to a (I J adaptively thresholded at step 152. A second step of the 

vertical scan line. The following discussion assumes that iterative process is to redefine the vertical dimensions of the 

when the image acquisition system is a one-dimensional window as the location of the vertical centroid, plus or 

CCD array, a vertical scan line represents the fast scan 35 minus a fixed predetermined distance of approximately 40 

direction and a horizontal scan line represents a slow scan pixels. A final step of this iterative process is to compute a 

direction. The following discussion further assumes that the new vertical centroid using the new window dimensions, 

spine 9 of bound document 10 is aligned approximately These steps are repeated until the number of non-zero pixels 

parallel to a vertical scan line, and approximately perpen- contributing to the centroid is unchanged. This iterative 

dicular to a horizontal scan line. 40 process has the effect of excluding outlier pixels and cen- 

At step 152, the image (IJ is adaptively thresholded to tering the starting point on the brightest section of the image 

identify pixels in the image (I 1 ) that represent light stripe 50 in the vicinity of the horizontal midpoint of the image (Ij), 

by eliminating non-uniform illumination across bound docu- which is assumed to be the light stripe, 

ment 10. The adaptive thresholding step 152 accounts for At step 156, the light stripe 50 is traced in the image (I J 

varying intensities between the light stripe 50 and the 45 in each horizontal direction one vertical scan line at a time 

background of the bound document 10 recorded across from the left and the right from the starting point found at 

image (Ij). Non-uniform illumination across image (Ij) step 154. (As defined above, a vertical scan line represents 

occurs because the imageable area is open to ambient light, the fast scan direction and a horizontal scan line represents 

and because the reflectance of a page of bound document 10 a slow scan direction of the scanning system 4.) Specifically, 

is unknown. Specifically at step 152, each vertical scan line 50 for each vertical scan line, pixels having a non-zero intensity 

(v) in image (I A ) is sequentially processed by transform value are collected within a neighborhood a small vertical 

generator 40 a scan line at a time. For a current vertical scan distance from a vertical location of the light stripe identified 

line being processed, each pixel in the current vertical scan in the previous scan line. The vertical location of the light 

line is assigned an intensity value of zero except for those stripe for a scan line is defined as the centroid or average in 

pixels having a brightness that is greater by a threshold 55 all dimensions of the pixels in the neighborhood. If no pixels 

amount than the brightness of each of the neighboring pixels have a non-zero intensity value in the neighborhood, the 

a fixed distance above and below it on the current scan line, vertical distance is increased by an incremental amount and 

and that have at least as great a brightness as the brightest the new neighborhood searched for pixels with a non-zero 

pixel along the current scan line, minus a tolerance amount. intensity value. If the neighborhood distance reaches a 

This thresholding step helps improve the contrast between 60 predetermined maximum value for a current scan line, then 

recorded ambient light and recorded light representing the the search for a vertical scan line location is abandoned and 

light stripe. the vertical scan line location of light stripe from the 

It will be understood by those skilled in the art that other previous vertical scan line is taken as the current vertical 

means exist to improve image contrast between recorded scan line location before processing a subsequent scan line, 

ambient light and light recorded from the light stripe pro- 65 Upon completing step 156, a trace of the light stripe 50 

jector 22. For example, contrast can be improved by pre- recorded in image (1^ which represent the contour of the 

filtering recorded light having the wavelength emitted by the surface of the bound document 10 is determined. 
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At siep 158, the trace determined at step 156 for light mined at steps 156, 158, and 160. By using a plurality of 

stripe 50 in image is transformed from pixel coordinates light stripes, a three dimensional contour of the surface of 

to height coordinates by calculating h b for each pixel as bound document 10 can be determined, 

defined by equation (1) set forth above. At step 160, the trace At step 166, the traces of any projected light stripes are 

is smoothed using a Gaussian smoothing kernel. Smoothing 5 converted to a page shape transform (T 2 ) such as a lookup 

is performed in order to average out the existence of large table. For example, FIG, 8 is a graph of traces 62 and 63 of 

dropouts or large transitions between vertical positions that the light stripes 5 and 6 shown in FIG. 1. The traces 62 and 

can exist in a trace of the light stripe. For example, the light 63 of light stripes 5 and 6 are defined in world coordinates 

stripe 50 in image (I.) can be broken at certain vertical (world -x, world- y, and world-z). The two light stripes 5 and 

positions because the light stripe is projected onto a printed 10 6 which are recorded as raw data in an image (II), are 

character on a page of document 10. Accordingly at step evaluated by page shape transform generator 40 to define 

156, if one or more pixels is not found to identify the light traces 62 and 63 that represent the distributions formed by 

stripe on a vertical scan line, then the position of the light stripe projection system 22 on bound document 10. The 

previous scan line is used to define the trace of light stripe traces 62 and 63 are traced, transformed, and smoothed as 

in image (IJ. As a result, large dropouts or jaggies caused 15 set forth in steps 156, 158, and 160 in FIG. 7. In addition, 

by pixels not defining the light stripe on particular scan lines if a spine region is detected in each profile 62 and 63, the 

are smoothed between other pixels forming the trace. regions is fit with a parametric model of a page shape as set 

At steps 162, the trace of the light stripe is searched to forth at step 164. After linear interpolation and integration of 

determine whether a bound document has a spine region 9. the traces 62 and 63, graphs shown in FIGS. 9, 10, or 11 are 

The spine region 9 of a bound document is defined herein as 20 generated. These graphs, which represent lookup tables, 

the location for which a "spininess" measure is at a maxi- provide mappings between the world coordinate system 

mum. The equation hj+hr-2h c is used to quantitatively (X,Y,Z) and the page coordinate system (page x, page y). 

measure the "spininess" of a vertical scan line (i.e. a scan FIG, 12 illustrates a two-dimensional geometry that can 

line along the fast scan direction), where h c , is the height of be used to track a location in world coordinates while 

the trace of the light stripe at a given vertical scan line, and 25 determining a location in page coordinates for light stripe 

h ( -, and h r are the height of the trace a fixed number of trace 62. Specifically, the trace 62 defines page height and 

vertical scan lines to the left and right, respectively, of the location in world coordinates in two dimensions. Using this 

location b c . This "spininess" measure is used to assess the geometry, values for page-x are determined by taking unit 

degree to which the spine of a bound document is centered steps Ax along the trace 62 while keeping track of the 

at a given vertical scan line in image (I t ) along the trace of 30 current location in world coordinates. In this example, 

the light stripe (i.e. centered at a position along the slow scan world-z values versus page-x values may be determined by 

direction). If the maximum "spininess" measure falls below linear interpolation, while world-x values versus page-x 

a preset threshold height then the trace of the light stripe is values may be determined using numerical integration of 

determined not to contain a book spine shape. steps 5 in the world coordinate system. 6 is defined as: 

If a spine is detected at step 162, a parametric model of 35 

page shape near the spine region is fit to the trace of the light 5 _ A* 2 ( 

stripe to improve the page shape estimate in the vicinity of N A* 2 + Az 2 
the spine region at step 164. This step is performed because 

the trace of the light stripe tends to be noisy in the spine where 6) AXj and Az are defined in FIG . 12. 

region. The spine region of the bound document more so 40 Subsequent to determining the lookup tables shown in 

than any other region along the trace of the light stnpe is not FIGS 9 _ u the lookup tables are re gi stcre d to insure correct 

well identified by the raw data in image (I,). To more positioning of the bound document between the image and 

accurately identify the spine region of a bound document m page c00rd i Dat e systems. The tables are registered because 

the trace of the light stripe, noisy raw profile data is replaced the origin of each coordinate system may not be property 

with a parabolic model on each side of the spine m image 45 aligned The registration of lookup tables shown in FIGS. 

(I,). The parabola is fit to the height of the trace of the light 9 _ u can be performed by referring to the location of the 

stripe at the spine location identified at step 162, and to the spjne crease detected at step 162 . Registering two lookup 

height and tangent of the smoothed profile at a fixed pre- lables m tWQ dimensions inv0IV es registering world-x versus 

determined number of vertical scan lines away from the page _ X) and world . z versus pagc - x i ookup ta bl cs . In this 

s P ine - 50 example, a location x in image coordinates that identifies 

Referring to the Appendix which includes Sections A-D, the crease of a trace of a light stripe found at step l62 is 

the portions of which are incorporated as part of the present mapped into a new location x rvv in world coordinates using 

description, detail an implementation in C source code of the the camera perspective transform ( Tl ) determined at step 

steps of FIG. 7. Section Aof the Appendix is source code for 142 ^ index int0 the trace of a hght slripe of the world 

providing general geometry support. Section B of the 55 x . origin (i e co 7 responding t0 world Xes0) is ^ wen by the 

Appendix is source code directed at the calibration of the expression- 
scanning apparatus. Section C of the Appendix is source 
code for estimating page shape. Section D of the Appendix 

is source code directed at capturing and de-warping images. x opp- x cpP -Xc»pp* 

It will be understood by those skilled in the art that steps 60 where "ppi" is the factor pixels-per-inch determined by the 
152, 154, 156, 158, 160, 162, and 164 can be performed on focal length of the image acquisition system 20 and the 
an image (1^) that has more than one light stripe recorded distance from the image acquisition system to the location 
therein. In the event a plurality of light stripes are recorded where the light stripe projects onto platform 8. The index 
in image step 154 is modified to detect a starting point x into the trace of the light stripe is transformed into an 
for each light stripe generated by projection system 22. 65 index x^^ in the world-x versus page-x, and world-z versus 
Subsequently for each starting point identified at step 154, a page-x lookup tables during the numerical integration pro- 
trace for each light stripe of bound document 10 is deter- cess of constructing lookup tables defining page shape 
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transform (T 2 ) shown in FIG. 12. More generally, the index 
x Q defines a shift that is used to correctly align the crease 
of the trace in the page coordinate system in the image 
coordinate system. Defining the shift requires that a refer- 
ence point, namely the crease of the trace in the image 
coordinate system, is found in the page coordinate system 
using the perspective transform (I J. However, if no crease 
is detected at step 162, the origin in the page coordinate 
system is assumed to be zero in the world coordinate system. 

FIG. 13 is a flow chart detailing the step 124 shown in 
FIG. 3. More specifically, FIG. 13 shows the steps required 
to reconstruct de- warped image (I 3 ) using a "polling" model. 
Specifically, using the "polling" model a location for each 
pixel in the de-warped image (I 3 ) is determined in the image 
(I 2 ). The de- warped image (I 3 ) is reconstructed by proceed- 
ing along an image in a vertical raster fashion, where the 
y-axis is the raster or fast scan direction. The color or gray 
density of each pixel in the de-warped image (I 3 ) is deter- 
mined by locating the position of that pixel in the recorded 
image (I 2 ), Once a position is located in the recorded image 
(I 2 ), depending on the location (i.e. how well it overlaps with 
the pixel boundaries in the recorded image), the polling 
operation may require access to several image pixels in the 
recorded image (LJ to reconstruct one output pixel in the 
de-warped image (I 3 ). 

At step 124, de-warped image (I 3 ) is formed by correcting 
image (l 2 ) using perspective transform (T\) and page shape 
transform (T 2 ). Specifically, at step 180 each page pixel 
location i(x,y) is mapped from the page coordinate system to 
the world coordinate system using the page shape transform 
T 2 to define target pixel locations i(X,Y,Z) in world coor- 
dinates. At step 182, these target pixel locations i(X,Y,Z) in 
the world coordinate system are mapped to the image 
coordinate system using the perspective transform 1\ to 
specify image pixel locations i(u,v). Transform T x is defined 
by the following perspective transform: 



x 




where: u\ v', and z 1 represent the coordinate system of the 
image acquisition system 20; z' is an intermediate variable 
that accounts for perspective depth effects; u=uV z 1 ; v=v7 z'; 
X, Y, and Z are location of a point of a bound document in 
world coordinates; and u, v are the location of a point of the 
bound document in image coordinates. At step 184, values 
specified at image pixel locations i(u,v) in recorded image I 2 
are interpolated to define values for each page pixel location 
i(x,y) in de-warped image (I 3 ). Interpolation is well known 
and can be performed using alternate schemes such as 
bi-linear interpolation, bi-cubic interpolation, and area map- 
ping. 

It will no doubt be appreciated that there are a number of 
possible manners in which to implement the various aspects 
of present invention. Unlike most scanning systems, the 
present invention is a bound document scanning system with 
a general imaging geometry. This aspect of the present 
invention provides a portable book scanning system with an 
imagable area that is not fixed. This facilitates movement of 
the bound document scanning system from one physical 
location to another, as well as one imagable area to another. 
In addition, this aspect of the invention obviates the need to 
upend a bound document between page turns as is required 
when placing the bound document in contact with a flat 
scanning surface. Consequently, the disclosed scanning sys- 
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tern has a maximum scanning rate which is bounded by the 
rate at which the pages of a bound document can be turned 
and recorded. 

It will also be appreciated that image data 32 and page 

5 shape data 34 can be recorded in a single original image that 
is later divided into two images using known image pro- 
cessing techniques. For example, red light can be projected 
by the light stripe projection system 22. Any red light can be 
removed from a recorded image using subtractive image 

]Q processing techniques. This subtractive technique defines 
image data and page shape data using the single original 
image. It will also be appreciated that the light stripe 
projection system is not limited to emitting red light so long 
as the wavelength of ambient light can be controlled, and 
that for the purposes of the present invention any two 

15 wavelengths that are distinguishable will do, such as green 
light and blue light. 

It will further be appreciated that scanning system 4 can 
be readily modified to incorporate functions of a system that 
automatically turns pages of a bound document. Examples 

20 of systems for tuning pages of bound documents are dis- 
closed by Turner et al., in U.S. patent application Ser. No. 
08/570,880 entitled "Bound Document Imager with Page 
Turner/' and U.S. patent application Ser. No. 08/570,791 
entitled "Bound Document Imager with Air Jet Page Turning 

25 System," both filed on Dec. 12, 1995 and assigned to the 
same assignee as the present invention. It will also be 
appreciated that scanning system 4 can be readily modified 
to include job level programming as described by Rourke et 
al. in U.S. Pat. No. 5,119,206. Job level programming can 

3Q include programming that manages input scanning as well as 
output rendering. Programming options for scanning bound 
documents include input commands for specifying whether 
to scan one or both sides of a bound document. Program- 
ming options for rendering scanned images of a bound 
document include specifying whether opposing sides of a 

35 bound document are to be printed on a common print media 
sheet or on separate print media sheets. 

Various aspects of the disclosed document scanning sys- 
tem may be readily implemented in software using software 
development environments that provide portable source 

40 code that can be used on a variety of hardware platforms. 
Alternatively, the disclosed system may be implemented 
partially or fully in hardware using standard logic circuits. 
Whether software or hardware is used to implement the 
system varies depending on the speed and efficiency require- 

45 ments of the system and also the particular function and the 
particular software or hardware systems and the particular 
microprocessor or microcomputer systems being utilized. 
The system, however, can be readily developed by those 
skilled in the applicable arts without undue experimentation 

50 from the functional description provided herein together 
with a general knowledge of the computer arts. 

The invention has been described with reference to a 
particular embodiment. Modifications and altera tioas will 
occur to others upon reading and understanding this speci- 

55 fication taken together with the drawings. The embodiments 
are but examples, and various alternatives, modifications, 
variations or improvements may be made by those skilled in 
the art from this teaching which are intended to be encom- 
passed by the following claims. 

60 I claim: 

1. In a scanning apparatus having an image acquisition 
system for recording images and a light stripe projector for 
projecting a light stripe, a method for determining a shape of 
the bound document positioned in an imaging area of the 
65 scanning apparatus, comprising the steps of: 

recording, with the image acquisition system, an image of 
the bound document having a light stripe projected 



05/17/2004, EAST version: 1.4.1 



5,835,241 



15 



16 



thereacross by the light stripe projector, the image 
being defined by an array of scan lines, with each scan 
line having pixels with an intensity value; 

adaptively thresholding the pixels forming the image to 
identify pixels defining part of the light stripe, said 
adaptive thresholding step compensating for non- 
uniform illumination across the bound document 
recorded by the image acquisition system; 

identifying a pixel starting location that forms part of the 
light stripe on a first scan line near a midpoint of the 
image; 

tracking, from the pixel starting location, a first set of 
pixels that further define the light stripe in the image, 
the first set of pixels representing a trace of the light 
stripe projected across the bound document; 

smoothing the first set of pixels to minimize large tran- 
sitions between scan lines; 

detecting a crease location along the first set of pixels that 
identifies a spine region of the bound document; and 

responsive to said detecting step, modifying the first set of 
pixels representing the trace, said modifying step 
replacing pixels near the crease location with a second 
set of pixels that are determined using a parametric 
model that approximates the spine region of the bound 
document. 

2. The method according to claim 1, wherein said step of 
adaptively thresholding the image includes the steps of: 

identifying pixels in the image with intensity values that 
vary in intensity from neighboring pixels by a threshold 
value, and 

setting the intensity values of those pixels in the image 
identified by said adaptive thresholding step to zero, 

3. The method according to claim 1, wherein said step of 
defining a starting location includes the step of identifying a 
neighborhood of pixels having a non-zero intensity value. 

4. The method according to claim 3, wherein said step of 
identifying a starting pixel location includes the steps of: 

defining a neighborhood of pixels in the image; and 
computing a centroid of the neighborhood of pixels. 

5. The method according to claim 4, further comprising 
the step of centering the neighborhood of pixels at a pixel 
location defined by the centroid. 

6. The method according to claim 5, further comprising 
the step of repeating said computing step and said centering 
step until the neighborhood of pixels does not change. 

7. The method according to claim 6, wherein said centroid 
is an average value of the intensity values in the neighbor- 
hood of pixels. 

8. The method according to claim 1, wherein said tracking 
step tracks in a first direction and a second direction one scan 
line at a time. 

9. The method according to claim 1, wherein said tracking 
step further comprises the steps of: 

identifying, within a predefined area in the image, pixels 
a small distance from the pixel starting location on a 
second scan line having a non-zero intensity value; and 

evaluating a centroid of the identified pixels with a 
non-zero intensity value on the second scan line. 

10. The method according to claim 9, further comprising 
the steps of: 

growing the predefined area; 

identifying, within the grown predefined area, pixels a 
small distance from the pixel starting location on the 
second scan line having a non-zero intensity value; and 

evaluating a centroid of the identified pixels with a 
non-zero intensity value on the second scan line. 
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11. The method according to claim 10, further comprising 
the step of abandoning said growing step at a predetermined 
maximum area. 

12. The method according to claim 1, further comprising 
the step of transforming the set of pixels from a pixel 
coordinate system to a height coordinate system. 

13. The method according to claim 1, wherein said 
smoothing step smoothes the set of pixels with a Gaussian 
smoothing kernel. 

14. The method according to claim 1, wherein said 
detecting step includes the step of assessing whether the 
trace contains a pixel that identifies the spine region of the 
bound document. 

15. The method according to claim 14, wherein said 
assessing step includes the steps of: 

measuring in the image changes in height between each 

scan line along the trace of the light stripe projected 

across the bound document; and 
selecting a pixel identifying a maximum measured change 

in height, the selected pixel identifying the spine region 

of the bound document. 

16. The method according to claim 1, wherein said 
modifying step includes the step of fitting a parabola to the 
pixels defining the trace of the light stripe in the spine region 
of the bound document. 

17. The method according to claim 1, wherein said 
detecting step identifies the crease pixel at a scan line in the 
image, 

18. The method according to claim 1, wherein said 
tracking step rejects pixels that are exist apart from pixels in 
the first set of pixels. 

19. A method for scanning a sequence of non-planar 
original images of a bound document, comprising the steps 
of: 

recording geometry data and raw image data for each 
image in the sequence of non-planar original images, 
said recording step including the steps of: 
projecting a light stripe on a non-planar original image, 
acquiring geometry data with an image acquisition 
system, and 

acquiring raw image data with the image acquisition 
system; and 

correcting the recorded raw image data for each image in 
the sequence of non-planar original images with the 
recorded geometry data, said correcting step 
performing, for each image in the sequence of non- 
planar original images, the steps of: 
identifying a pixel starting location that forms part of 

the light stripe on a scan line near a midpoint of the 

geometry data, 
tracking, from the pixel starting location, a set of pixels 

that further define the light stripe in the image, 
smoothing the set of pixels to minimize large transi- 
tions between scan lines, 
converting the set of pixels, which represent a trace of 

the light stripe projected across the bound document, 

to a page shape transform, and 
de-warping the recorded image data with the page 

shape transform to provide corrected image data. 

20. The method according to claim 19, wherein said 
recording step is completed before initiating said correcting 
step. 

21. The method according to claim 19, further comprising 
the step of storing the geometry data and the raw image data 
in a memory. 

22. The method according to claim 21, wherein said 
correcting step is delayed until the raw image data is 
displayed on a terminal. 
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23. The method according to claim 21, wherein said 
correcting step is delayed until Ihe raw image data is 
reproduced as hard copy. 

24. The method according to claim 19, further comprising 
the step of recording calibration data with the image acqui- 
sition system. 

25. The method according to claim 24, wherein said 
correcting step includes the step of determining a perspec- 
tive transform with the calibration data, the perspective 
transform defining a relationship, between a two- 
dimensional coordinate system of the image acquisition 
system and a three-dimensional coordinate system of the 
bound document. 
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26. The method according to claim 25, wherein said 
de- warping step includes the step of transforming the image 
data with the perspective transform. 

27. The method according to claim 19, wherein said 
recording step and said correcting step are performed in 
parallel for each image in the sequence of non-planar 
original images. 

28. The method according to claim 19, wherein said 
recording step and said correcting step are performed 
sequentially for each image in the sequence of non-planar 
original images. 
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